Missing data techniques for robust speech recognition

In noisy listening conditions, the information available on which to base speech recognition decisions is necessarily incomplete: some spectro-temporal regions are dominated by other sources. We report on the application of a variety of techniques for missing data in speech recognition. These techniques may be based on marginal distributions or on reconstruction of missing parts of the spectrum. Application of these ideas in the Resource Management task shows performance which is robust to random removal of up to 80% of the frequency channels, but falls off rapidly with deletions which more realistically simulate masked speech. We report on a vowel classification experiment designed to isolate some of the RM problems for more detailed exploration. The results of this experiment confirm the general superiority of marginals-based schemes, demonstrate the viability of shared covariance statistics, and suggest several ways in which performance improvements on the larger task may be obtained. The missing data problem arises naturally in many pattern recognition tasks [2,8] where elements of data vectors to be classified are unavailable during training and/or recognition. The causes of incomplete evidence include unreliable sensors, band-restricted data transmission (e.g. the spectral filtering action of a telephone channel), or partial occlusion of the desired pattern by an interfering signal. In the latter case, it is assumed that some preprocessor is able to determine which parts of the mixed observation correspond to the source to be classified. Our motivation for studying the missing data problem derives from ongoing studies at Sheffield and elsewhere [1] on computational auditory scene analysis (CASA), in which evidence for different sound sources is separated using auditory grouping principles. CASA is an attractive paradigm for robust ASR. It makes no assumptions about the type and number of acoustic sources which make up the mixture, and does not require prior exposure to these sources. However, separation will never be able to recover all the evidence: there will be some regions where other sound sources dominate. CASA-based robust ASR requires that the resulting missing data problem be confronted. In previous work [4,9] we demonstrated that it is possible to remove high proportions (up to 90%) of the input spectrum without significant deterioration in recognition rates. In ICASSP-95, we reported (using NOISEX) noise tolerance comparable to that of human listeners when only those spectro-temporal regions with a favourable local SNR were retained. Subsequently, we have applied missing data techniques to the Resource Management (RM) …